from a3py import *
import scipy as sp
In this notebook we are going to perform audio denoising by spectral subtraction. The idea behind spectral subtraction is:
You will make heavy use of the helper class a3py.STFT, which has been provided for you.
Let's start off by opening an audio file. I've provided an example audio file, ./noisyhellocs6682, for you to testt on, but I want you to hand in the result of spectral subtraction on your own audio for your submission. Specifically, use the notebook ./MySpectralSubtraction.ipynb to generate what you will be handing in.
a = Audio('./noisyhellocs6682.wav');
a.play()
a.showSpectrogram();
To estimate the noise spectrum, we want to pick out a span of time from the original audio that only has background noise. An easy way to do this is to look at the spectrogram for a start and end time with no speech in between.
When you run this on your own sound, be sure to record a few seconds of background noise for this purpose.
noise_start = 8.5;
noise_end = 12;
def showNoiseClip(audio, noise_start, noise_end):
fig, axs = plt.subplots(nrows=1,ncols=2,figsize=[12,4]);
orig_ax = axs[0];
noise_ax= axs[1];
audio.showSpectrogram(fig = fig, ax=orig_ax);
orig_ax.set(title='Original Signal')
lineargs = dict(linewidth=3, color='green');
orig_ax.axvline(x=noise_start, **lineargs);
orig_ax.axvline(x=noise_end, **lineargs);
noise = audio.GetClip(noise_start, noise_end);
noise.showSpectrogram(fig=fig, ax=noise_ax);
noise_ax.set(title='Noise Spectrogram')
print("Selected Noise Audio:")
noise.play()
showNoiseClip(a, noise_start, noise_end)
Selected Noise Audio:
In our sample audio, 8.5s-12s is just background noise, so we will use that. We can use the helper function audio.GetClip(start, end) to clip our audio to a new Audio object.
noise = a.GetClip(noise_start,noise_end);
noise.play();
noise.showSpectrogram();
Complete the following two functions to implement spectral subtraction. My implementation is 5 lines of code---yours may be more than that, but try not to make it too complicated.
def estimatePerFrequencyNoiseMagnitude(noise_stft):
# estimate the magnitude of noise at each frequency in a noise-only stft.
# the return value should be an np.ndarray of lenth equal to the number of frequencies in noise_stft
# taking the average magnitude for each frequency should do fine...
raise NotImplementedError;
def GetSTFTDenoisedWithSpectralSubtraction(input_stft, estimated_noise_magnitudes):
# Return a copy of the input_stft that has been denoised with spectral subtraction using the noise magnitude estimates in estimated_noise_magnitudes.
# Remember not to end up with negative magnitudes.
# calculate phase and magnitudes, and set them using setPhaseMag like so:
# rval = input_stft.clone();
# rval.setPhaseMag(phase=, mag=);
# return rval;
raise NotImplementedError;
def Denoise(input_stft, noise_stft):
#estimate the noise spectrum
noise_spec = estimatePerFrequencyNoiseMagnitude(noise_stft);
#input stft denoised with estimate based on noise
stft_denoised = GetSTFTDenoisedWithSpectralSubtraction(input_stft, noise_spec);
#denoised audio
denoised = stft_denoised.GetISTFTAudio();
return denoised;
def displayResults(input_audio, noise_start, noise_end):
noise_clip = input_audio.GetClip(noise_start, noise_end);
#input stft
input_stft = STFT.FromAudio(input_audio);
#noise stft
noise_stft = STFT.FromAudio(noise_clip);
denoised = Denoise(input_stft, noise_stft);
stft_denoised = STFT.FromAudio(denoised);
fig, axs = plt.subplots(nrows=2,ncols=2,figsize=[10,6]);
orig_ax = axs[0][0];
noise_ax= axs[0][1];
denoised_ax = axs[1][0];
noisespec_ax = axs[1][1];
input_stft.showPower(fig = fig, ax=orig_ax);
orig_ax.set(title='Original Signal')
lineargs = dict(linewidth=3, color='green');
orig_ax.axvline(x=noise_start, **lineargs);
orig_ax.axvline(x=noise_end, **lineargs);
noise_stft.showPower(fig = fig, ax=noise_ax);
noise_ax.set(title='Noise Sample ({}s-{}s)'.format(noise_start, noise_end))
denoised.showSpectrogram(fig=fig, ax=denoised_ax);
denoised_ax.set(title='Denoised Signal');
plt.plot(stft_denoised.freqs, estimatePerFrequencyNoiseMagnitude(noise_stft));
noisespec_ax.set(title='Estimated Noise Spectrum');
print("Original:")
a.play();
print("Denoised:")
denoised.play()
displayResults(a, noise_start, noise_end)
Original:
Denoised:
Run the same process below on some audio that you record.
myaudio = Audio('./noisyhellocs6682.wav');
myaudio.play()
my_noise_start = 8.5;
my_noise_end = 12;
showNoiseClip(myaudio, my_noise_start, my_noise_end)
Selected Noise Audio:
displayResults(myaudio, my_noise_start, my_noise_end)
Original:
Denoised:
Run this entire notebook when you have completed it (with your implementation and denoised audio results included). Download the completed notebook as HTML, and submit it on CMS.